AndroidTools

Contents

This page is a gathering place for information about the android-tools packaging team, which is focused on packaging the Android development tools for Debian. There are also some packages which help run Debian in a chroot on Android. The goal of this team is to get as much of the Android SDK and development tools into Debian as possible. There are many advantages to having the SDK and tools in Debian, rather than relying only on the Google distributions:

trivial install for specific tools, like adb, fastboot, etc.

To read more about the rationale behind this work, see this blog post: https://guardianproject.info/2015/04/30/getting-android-tools-into-debian/

To communicate with this team, join our low traffic mailing list, [email protected] and on the IRC channel #debian-android-tools (webchat). You can also join the IRC Channel through Matrix

The binaries for the Android SDK downloadable from Google have a proprietary license but the source code is free software so Debian is packaging it. Not all Android SDK packages can be installed from Debian, some never will be in Debian because they are too specific to Android. Sylvain Beucler's libre Android rebuilds and/or Google's non-free binaries can also be used with the Debian Android SDK.

If you're just starting out with building apps, we suggest that you first read the Introduction to build packages with Debian's Android SDK.

If you are already familiar with how to use these tools, you might want to look these brief instructions below. Here are the steps for building Android apps using Debian's Android SDK on Stretch.

In build.gradle, change compileSdkVersion to 23 and buildToolsVersion to 24.0.0

run gradle build

The Gradle Android Plugin is also packaged. Using the Debian package instead of the one from online Maven repositories requires a little configuration before running Gradle. In the buildscript {} block:

add maven { url 'file:///usr/share/maven-repo }` to repositories

use compile 'com.android.tools.build:gradle:debian' to load the plugin

Currently there is only the target platform of API Level 23 packaged, so only apps targeted at android-23 can be built with only Debian packages. We will add more API platform packages via backports afterwards. Only Build-Tools 24.0.0 is available, so in order to use the SDK, build scripts need to be modified. Beware that the Lint in this version of Gradle Android Plugin is still problematic, so running the :lint tasks might not work. They can be turned off with lintOptions.abortOnError in build.gradle. Google binaries can be combined with the Debian packages, for example to use a different version of the platform or build-tools.

In stretch-backports (and soon testing), the Gradle Android Plugin is patched to work with Debian's Android SDK. It detects what versions of API Levels and Build-Tools are available and you no longer need to modify the build scripts. In order to build apps, do the following:

Thus, init.gradle forces Gradle to use the Gradle Android Plugin in Debian and the plugin will do the rest.

Here's an example project: https://gitlab.com/Matrixcoffee/hello-world-debian-android

It is possible to install the proprietary binary packages to run the Android emulator on Debian. These package binaries have a proprietary license, but are built from free software sources. It isn't yet confirmed that they are built from 100% free software source code, but they seem to be.

There are a number of ways that people working on packaging Android Tools communicate. Here is a list:

blog: https://android-tools-team.pages.debian.net/blog/

main chat room: #debian-android-tools OFTC IRC [matrix] (webchat).

main email list: [email protected]

IRC chat room for automated messages: #debian-android-tools

some Java packages are essential to Android, so we also talk here: #debian-java (webchat).

We want make it as easy as possible for anyone to get involved in the Android Tools Team, and contribute in ways that they think are important. It is important to note that some of the packages here are quite complicated, and the Android SDK is a very large project. That means that changes must be discussed with the team before they can be committed and pushed. That discussion can happen in anywhere, including merge requests.

So here is the Android Tools Team policy for granting membership to the team, based on the GitLab user roles:

requests from anyone else must be posted to https://salsa.debian.org/android-tools-team/admin/issues and are approved by "lazy consensus"

specific requests for "Master" level must also be posted to https://salsa.debian.org/android-tools-team/admin/issues and are approved by "lazy consensus"

This is based on "Lazy consensus": if no one objects to a request within one week or so, then any Master/Owner is free to grant the request. So silence is considered approval, but not automatic approval. An existing Master/Owner must actually grant the request after the waiting period.

Mostly, our procedures are about which packages are standalone and can be freely uploaded, versus which packages are interwoven and must be uploaded in coordinated batches (e.g. the Android SDK). repo or enjarify is a standalone package, for example. The interwoven ones basically all follow the Android SDK git repo naming scheme for the source package names:

android-platform-*

android-framework-*

android-sdk-meta

The naming scheme for android-tools packages is as follows:

source package names are named after the git repository, prefixed by android- (e.g. android-platform-system-core, android-platform-system-extras, android-platform-build, android-platform-frameworks-base, etc.)

binary packages of utilities that run on Debian directly are named after the utility itself (e.g. zipalign, aapt, etc.)

shared libraries that are only used by android-tools packages are named after the library, without a ABI version number in the package name, and prefixed by android-. (e.g. android-libhost, android-libcutils-dev, etc. In the Google builds, these are built as static libraries, and linked statically into each binary. In the Debian builds, they are built as shared libraries and installed into /usr/lib/android.

The structure of each source package is documented in the README.source of each source package for this team:

https://salsa.debian.org/android-tools-team/android-permissions/tree/master/debian/README.source

https://salsa.debian.org/android-tools-team/android-platform-build/tree/master/debian/README.source

https://salsa.debian.org/android-tools-team/android-platform-frameworks-base/tree/master/debian/README.source

https://salsa.debian.org/android-tools-team/android-platform-frameworks-native/tree/master/debian/README.source

https://salsa.debian.org/android-tools-team/android-platform-system-core/tree/master/debian/README.source

https://salsa.debian.org/android-tools-team/android-platform-system-extras/tree/master/debian/README.source

The packages in this team are structured somewhat unusually because we are trying to keep the source packages as close as possible to the upstream source organization while still working in a Debian way. Google builds the Android OS and SDK as one giant thing, something like 10 gigs of source code. But the code is broken up into many different git repos that are coordinated using the Android team's tool called repo. Also, there are lots of shared libraries used between the various Android SDK tools, but since everything is always built together, those shared libraries are unversioned.

All this means that it is essential that any Android SDK package is only built against the exact same version of all its Build-Depends and only uses the exact same version of an Android SDK package as a Depends. To achieve that, we use the substvars variable in dependency declarations: (>= ${source:Version}).

Additionally, because of this and the circular dependencies, it is important to upload updates in the correct order. Some packages also have to be uploaded using a multi-stage method. Here is the estimated upload order:

android-platform-dalvik (Needs android.jar of the latest API Level)

Each packages belonging to the same stage are unrelated to each other and can be uploaded in any order.

This order only represents one possible way of updating all of them to a new major version. In practice, you can delay or advance any packages as long as the build-dependencies satisfy.

Tools in Android SDK come from various repositories. Here is a list of tools consisting the entire Android SDK, with each tool followed with the corresponding upstream repository name.

ddms: platform/tools/swt (deprecated)

hierarchyviewer: platform/tools/swt (deprecated)

traceview: platform/tools/swt (deprecated)

git clone platform/tools/base

source code studio-master-dev/apkparser/analyzer/

git clone platform/tools/base

source code sdklib/internal/avd/

git clone platform/external/r8

source code android/tools/r8

draw9patch: platform/tools/base

emulator: platform/sdk

git clone platform/tools/base

source code studio-master-dev/jobb/

git clone platform/system/tools/hidl

source code refs/heads/master

git clone platform/tools/base

source code studio-master-dev/lint/

mksdcard: platform/sdk

git clone platform/tools/swt (deprecated)

source code master/monkeyrunner/ (deprecated)

proguard: https://github.com/Guardsquare/proguard (Already in Debian)

screenshot2: platform/tools/base

git clone platform/tools/base

source code sdklib/tool/sdkmanager/

uiautomatorviewer: platform/tools/swt (deprecated)

adb: platform/system/core

dmtracedump: platform/art

etc1tool: platform/development

fastboot: platform/system/core

hprof-conv: platform/dalvik

systrace.py Catapult Project

aapt/aapt2: platform/frameworks/base

aidl: platform/frameworks/base

bcc_compat: platform/frameworks/compile/libbcc

dexdump: platform/art

dx: platform/dalvik

jack: toolchain/jack

jill: toolchain/jill

llvm-rs-cc: frameworks/compile/slang

mainDexClasses: platform/dalvik

apksigner: platform/tools/apksig

split-select: platform/frameworks/base

zipalign: platform/build

dexdump apktool gplaycli androguard enjarify dmtracedump

Gradle Plugin for Android: platform/tools/base

repo: tools/repo

signapk: platform/build

signtos: platform/build

ziptime: platform/build

There are many naming schemes for versions in Android, and none are used everywhere. For the Android OS itself, there are three schemes: version names, like Gingerbread or Kitkat; version numbers, like 2.3.7 or 4.4.2; and, "API level" numbers, like android-10 or android-19. None of these line up with each other. For example, the Jelly Bean name spans 4.1 through 4.3.1 and android-16 through android-18. The version numbers and API level also do not line up, with android-14 covering 4.0.1 - 4.0.2, android-15 covering 4.0.3 - 4.0.4, but android-16 covers all of 4.1.x.

The madness doesn't end there. Next up we have SDK version numbers, which are like 20, or 23.0.2. Part of the SDK includes the build-tools package, which has its own version numbers, which are like 18.1.1 and 20.0.0, but these do not line up with the SDK version numbers. The platform-tools seems to have its own versioning scheme also, but it is not really exposed. Then there are the NDK release numbers, which are like r8b or r10, and they also do not line up with anything else.

In the git repo, there are a mishmash of tags and branches that do not represent all of these various versions. There does seem to be consistent tagging of the OS releases, you can see those listed under Source Code Tags and Builds. There are a few branches that seem to line up to the SDK version numbers, like tools_r21 and tools_r22.2, but there is not a complete set of those branches.

Android SDK Tools seems to have it's own versioning scheme, since the major version is ahead of the SDK major version:

Android SDK Build-tools seems to follow the SDK major version, but have its own minor and micro versions:

Android SDK Platform-tools seems to follow the SDK major version only.

You can find the official Google package names and versions in the index XML files that the android tool downloads when doing updates:

https://dl.google.com/android/repository/repository-10.xml

https://dl.google.com/android/repository/repository-11.xml

https://dl.google.com/android/repository/repository-12.xml

https://dl.google.com/android/repository/repository2-1.xml

https://dl.google.com/android/repository/addon.xml

https://dl.google.com/android/repository/addon-6.xml

https://dl.google.com/android/repository/sys-img/android/sys-img.xml

https://dl.google.com/android/repository/sys-img/x86/addon-x86.xml

However, the exact version number of Android SDK toolsets can be found in a source.properties under each directories. Here are the source locations of each version declaration:

SDK Tools: https://android.googlesource.com/platform/sdk/+/master/files/tools_source.properties

SDK Platform-tools: https://android.googlesource.com/platform/development/+/master/sdk/plat_tools_source.prop_template

SDK Build-tools: https://android.googlesource.com/platform/development/+/master/sdk/build_tools_source.prop_template

Gradle plugin: https://android.googlesource.com/platform/tools/buildSrc/+/master/base/version.gradle

Which matches the official release notes.

Note that according to the source code the major version number of Platform-tools and Build-tools is simply the API Level, which is defined here. But for SDK Tools it follows its own version pattern and does not relate to the API Level.

Some tools tracks their own version and do not follow the SDK version:

This team started with the android-tools package, which is a manually assembled collection of source code needed to build adb, fastboot, etc. This approach is not maintainable in the long run, especially as we work to add the whole Android SDK. So the android-tools package is deprecated, and will be replaced by the collection of packages described above.

Packaging all of these Android SDK Tools is so complicated because the source code is organized into semi-arbitrary "projects" that are split up between many different git repos. Those git repos are then all checked out at the same time using Google's crazy repo tool. Then the entire Android OS and SDK are built using a single, unified build system that requires something like 8 gigs of source code be downloaded. On top of that, one of the more confusing aspects of this whole project is that the same source code base is used to build the OS and the SDK. Those two can start with the same source code files, and end up with very different resulting binaries.

The Android SDK has a number of private libraries that are shared between many different utilities that are part of the Android SDK. Upstream statically links them into each executable. That didn't seem very Debian-ish, so instead they are structured as private shared libraries. Nothing should link to these android-lib* shared libraries that is not part of the Android source. This private shared library arrangement allows for security patches in the android shared code to be applied without having to rebuild everything.

All of the android-* packages must be updated to the latest version at the same time. It is more unpredictable to have android-* packages at different upstream versions since no one is running that configuration, and upstream does not do anything to support that (e.g. no versioned ABI, etc). Therefore, the ABI is guaranteed to be compatible since they'll all be built together. In practice, this means that the process of updating to the latest upstream version has to start from the most core packages, then progress to the ones that depend on it.

A single source package might make some things easier, but it would be about 8-12 gigs in size. That would make it very difficult for many people to work with that source package.

There is lots left to do before someone can do Android development using only official Debian packages. The best way to get started is to join the email list and ask there. We're also in IRC at #debian-android-tools. Here are some general topics that need work:

build an app using only the Debian packages for gradle, gradle plugins, build tools and platform tools (i.e. apt install android-sdk then gradle build), then file bug reports on anything that isn't working

figure out packaging scheme for various android "platforms", i.e. /opt/android-sdk/platforms/android-* and /opt/android-ndk/platforms/android-*

figure out how to represent Android NDK clang "toolchains" in Debian, i.e. /opt/android-sdk/toolchains

fdroidserver is the tool suite for managing FDroid app repos and making release builds. It uses the Android SDK and NDK to make the builds and work with APKs. There are three notable milestones for fdroidserver packaging needs:

everything to create and manage app repos DONE!

Here are a couple of easy tasks to do that will further this effort along:

build Android apps using apt install android-sdk on stretch and newer

package these existing SDK packages for other distros (Arch, Fedora, NixOS, Homebrew, ?MacPorts, Cygwin, etc)

get answer from Google on how the SDK versions are marked in the source git repos (i.e. what tag/branch is SDK v23.0.2? And what tag/branch is build-tools v20.0.0?)

Is https://source.android.com/source/code-lines.html#terms-and-caveats any helpful? Among other things, it says:

A release corresponds to a formal version of the Android platform, such as 1.5, 2.1, and so on. Generally speaking, a release of the platform corresponds to the version in the SdkVersion field of AndroidManifest.xml files and defined within frameworks/base/api in the source tree.

seems to vaguely contribute to understanding, but no concrete answers for which commit ID to build for SDK release v23.0.2. -- HansChristophSteiner 2014-10-07 19:51:57

android-platform-system-extras package, replacing android-tools-fsutils package.

update all source packages to the latest SDK release (v23.0.2, provided you can find the right tag or branch)

Jack Deprecated

Android Support Library

The rest of an "Android platform" ("android.jar" is done).

android-platform-dalvik needs update to include "shrinkedAndroid.jar" generated with "android.jar"

android-platform-tools-swt and probably though unlikely android-platform-tools-buildsrc

android-platform-frameworks-compile-libbcc for bcc_compat, llvm-rs-cc, etc.

For lack of a better name, this basically contains some of the things we have learnt and some of the general ways in which we work with android packages.

Fedora package for android-tools

Fedora package for abootimg

Arch Linux packages up the Google binary packages https://wiki.archlinux.org/index.php/Android

CategoryPackaging